Fukumizu : Statistical Active Learning in Multilayer Perceptron 3

|This paper proposes new methods of generating input locations actively in gathering training data, aiming at solving problems special to multilayer perceptrons. One of the problems is that the optimum input locations which are calculated deterministically sometimes result in badly-distributed data and cause local minima in back-propagation training. Two probabilistic active learning methods, which utilize the statistical variance of locations, are proposed to solve this problem. One is parametric active learning and the other is multi-point-search active learning. Another serious problem in applying active learning to multilayer per-ceptrons is the singularity of a Fisher information matrix, whose regularity is assumed in many methods including the proposed ones. A technique of pruning redundant hidden units is proposed to keep the regularity of a Fisher information matrix, which makes active learning applicable to multilayer perceptrons. The eeectiveness of the proposed methods is demonstrated through computer simulations on simple artiicial problems and a real-world problem in color conversion.